web stats
 
   
 
Selected Research Project
 
Project Title Integrating a New Mixture Item Selection Approach with Content Balancing Methods in Computerized Adaptive Testing
 
Principal Investigator Dr Leung Chi Keung, Eddie
 
Area of Research Project
Educational Measurement and Assessment
 
Project Period
From 10/2009 To 6/2011
Objectives

This project proposes scientific ways of integrating a new item selection approach with content balancing methods for tackling different conditions of content requirement in computerized adaptive testing.

Methods Used

This research will be carried out through simulation studies:

  • Item pool:
    • Two item pools will be used. The first mimics the structure of a previous pool of 420 items for a mathematics assessment. The second mimics the structure of 700 items for a professional degree entrance exam.
  • Pool size:
    • For each set of constraints, tests will be adaptively constructed from each of the two pools using (i) full size and (ii) about 75% of items.
  • Target maximum exposure rate:
    • (i) 0.2 and (ii) 0.3.
  • Test-length:
    • Around 25 items and 40 items.
  • Evaluation:
    • The performance of each advanced selection method will be evaluated in terms of measurement efficiency, item security, pool utilization and content balancing.

Summary of Findings

New advanced item selection methods for addressing item security, pool utilization and content balancing

The topic under investigation had prospective practical significance in that it can potentially widen CAT’s range of application. It is anticipated that an increasing number of local and international organizations may offer their traditional paper-and pencil tests in the form of computerized testing formats when they are convinced that the item-selection method reduces the time required for taking the test and provides valid and accurate test results. With appropriate computerized adaptive testing systems schools can collect timely and reliable information on individual performance to cater for individual needs.

Output
  • Mixed Approach for Item Selection in e-testing, Research and Practice in Technology Enhanced Learning, vol 5, pp3-17
  • A Comparison between Fixed and Flexible Content Balancing Methods in Computerized Adaptive Testing. Paper presented at the 7th Conference of the International Test Commission 2010.
  • How would Mixed Item Selection Approach work with Weighted Deviation Model and Shadow Test Assembly for Constrained Adaptive Testing? Paper presented at the 76th Annual and the 17th International Meeting of the Psychometric Society 2011.
Biography of Principal Investigator
Dr. Eddie Chi-keung LEUNG is an Assistant Professor at the Department of Mathematics and Information Technology. He obtained his bachelorˇ¦s degree from the University of Hong Kong, masterˇ¦s degree from Macquarie University, and Doctor of Philosophy from the Chinese University of Hong Kong. Dr. Leung has extensive teaching experience, both in the secondary and tertiary levels. In addition, he has supervised several undergraduate and postgraduate students in projects on mathematics education. His research interests include problem solving, cognitive and affective aspects of mathematics education, and computerized mathematics assessment. Dr. Leung has been the principal/sole investigator of numerous research and development projects supported by the Institute. He was financially supported by the Quality Education Fund (QEF) for the project ˇ§Promoting Problem Solving Practice among Teachers and Pupils in Primary Mathematicsˇ¨ in 2004. This project was awarded the Certificate of Merit by the QEF. He also successfully obtained a prestigeous grant from the competitive General Research Fund. He has published numerous books and journal articles on problem solving, computerized mathematics assessment, and general mathematics education.
Funding Source
General Research Fund